Implicit Bilevel Optimization: Differentiating through Bilevel Optimization Programming

Bilevel Optimization Programming is used to model complex and conflicting interactions between agents, for example in Robust AI or Privacy preserving AI. Integrating bilevel mathematical programming within deep learning thus an essential objective the Machine Learning community. Previously proposed approaches only consider single-level programming. In this paper, we extend existing optimization propose Differentiating through (BiGrad) end-to-end of models that use as a layer. BiGrad has wide applicability can be modern machine frameworks. applicable both continuous combinatorial problems. We describe class gradient estimators case which reduces requirements terms computation complexity; variable, takes advantage push-back approach (i.e. vector-jacobian product) efficient implementation. Experiments show successfully extends Programming.